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Preface 


This  thesis  is  the  presentation  of  the  results  of  ?n  extensive 

literature  search  to  discover  applications  of  Bayes  formula  for  the 
solution  of  typical  decision  problems  encountered  by  reliability  and 
maintainability,  R/M,  engineers. 

The  purpose  of  this  study  is  to  present  an  overview  of  the  basic 
elements  and  fundamental  concepts  of  general  decision  theory  and  show 
how  they  might  be  useful  to  the  R/M  practitioner.'  It  is  hoped  that  this 
brief  exposure  to  some  of  the  organized  and  systematic  techniques  to 
decision-making  will  create  a  thirst  for  more  detailed  individual 
investigation  for  ways  to  treat  similar  situations  arising  in  both 
personal  and  professional  life. 

My  most  sincere  thanks  are  extended  to  my  Thesis  Advisor,  Professor 
A.  H.  Moore  for  his  inspiration,  motivation,  and  consultation.  I  am 
also  deeply  indebted  to  my  whose  degree  of  belief  in  me 

was  great  enough  to  assign  a  prior  subjective  probability  of  thesis 
completion  equal  to  one.  Bayes  rule  always  works  with  a  driving  force 
like  that. 


Lewis  R.  V.’hite 
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Abstract 

Bayesian  statistics  provide  the  necessary  mathematical  techniques 
to  pool  all  available  subjective  and  experimental  information  when 
estimating  reliability.  The  uncertainties  associated  with  analytical 
predictions  or  limited  test  data  considered  separately  are  significantly 
reduced  when  these  two  sources  of  information  are  combined.  The 
introduction  .of  judgement  and  pertinent  engineering  theory  and 
experience  to  qualify  point  estimates  is  the  key  to  realistic  and 
practical  solutions  to  decision  problems  in  which  reliability  is  a 
primary  consideration. 

A  method  for  periodic  reliability  assessment  is  presented.  A 
hypothetical  example  is  used  to  show  how  iterative  inferences  on 
system  reliability  can  be  drawn  from  initial  estimates  of  unit/subsystem 
reliability  and  heterogeneous  time  and  failure  data  accumulated  during 
various  stages  of  design  verification,  electrical  performance, 
environmental,  etc.,  testing.  Sample  worksheets  for  recording  inputs 
required  for  the  assessment  technique  are  provided  as  an  appendix. 
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BAYESIAN  RELIABILITY "ASSESSMENT 
FOR  SYSTEM  PROGRAM  DECISIONS 

I.  Introduction 

The  number  of  cost-plus  type  government  contracts  which  permit 
generous  budgets  for  expensive  reliability/maintainability  studies 
and  tests  to  minimise  risk  is  dwindling.  The  time  has  come  when  both 
government  and  industry  have  been  forced  to  tighten  their  money  belts. 
The  resulting  squeeze  has  made  backbones  staffer  and  eyes  more  pointed. 
In  today's  austere  atmosphere,  what  used  to  be  accepted  techniques  and 
practices  are  challenged  as  potential  candidates  for  modification  and 
revision  in  an  attempt  to  reduce  costs. 

But  although  there  is  less  money  to  spend,  the  -requirements  are 
just  as  stringent,  if  not  more  so.  The  impact  of  this  current  climate 
is  felt  throughout  all  organizational  levels  in  the  military-industrial 
complex.  Essentially,  practicality  and  utility  are  the  keynotes  that 
are  replacing  desirability^  feasibility  and  availability.  The 
reiiability/maintainability  practitioner  must  adjust  to  this  new 
environment. 

The  theme  of  getting  the  most  for  the  money  is  really  nothing  new. 
However,  the  recent  reduction  in  resources  has  resulted  in  more 
scrutiny  of  actions  taken  to  achieve  this  end.  No  longer  is  a  sole 
recommendation  on  a  particular  issue  accepted  at  face  value.  The 
following  simple,  straightforward  and  sensible  questions  are  being 
asked  more  frequently: 
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a.  Is  this  the  only  way' to  solve  the  probiem? 

b.  Were  other  solutions  considered? 

c.  Why  is  the  proposed  solution  considered  the  best? 

Decision  theory  provides  the  framework  for  systematically  addressing 
these  questions  and  quantifying  the  uncertainties  associated  with  their 
answers.  The  primary  purpose  of  this  thesis  is  to  present  a  method 
for  periodic  assessment  of  system  reliability  risk  based  only  on 
analytical  predictions  and  limited  usage  experience.  Particular 
emphasis  is  placed  on  the  use  of  subjective  probability  and  Bayes’ 
formula.  These  techniques  have  received  much  attention  in  recent  years 
and  considerable  literature  is  available  for  those  who  wish  to  pursue 
more  advanced  analyses  than  the  one  presented.  As.  an  aid  to  those  who 
desire  more  detail,  an  additional  study  objective  is  to  provide  a 
selective,  listing  of  references  in-  which  general  decision  theory  and_ 
Bayesian  ,ar.alys  es  are  used  to  solve  .problems  of  choice  usually  en¬ 
countered  in  the  field  of  reliability  and  maintainability.  These 
techniques  are  considered  to  be  valuable  analytical  tools  for  assessing 
alternatives  during  the  establishment,  specification,  verification  and 
demonstration,  .of  quantative  reliability/naintainability  requirements 
throughout  a  system's  life.. 

To  achieve  study  objectives,  a  comprehensive  survey  of  recent 
literature  was  conducted  with  the  aid  of  abstracting,  indexing,  and 
search  services'.  The  results  of  this  survey  and  the  subsequent  study 
are  presented  as  follows:  Section  II  summarizes  the  elements,  concepts 
and  notation  associated  with  general  decision  theory;  Section  III 
introduces  Bayes’  formula  and  outlines  how  it  may  be  used  to  combine 
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all  available  relevant  information.,  both  subjective  judgements  and 
objective  data,  in  the  decision  process;  Section  IV  addresses  the 
selection  of  prior  distributions  applicable  to  the  reliability  assess¬ 
ment  problem;  Section  V  outlines  a  proposed  method  for  applying  Bayesian 
techniques  to  assess  the  reliability  of  a  system  prior  to  formal  testing 
Section  VI  provides  a  discussion  of  the  assessment  technique  with  a  few 
refinements;  and  Section  VII  contains  the  conclusions  and  recommendation 
of  the  study.  A  supplementary  bibliography  is  provided  as  an  appendix. 
Listed  are  both  references  which  are  probably  directly  applicable  to  the 
thesis  but  were  not  acquired,  and  also  those  which,  are  indirectly  or 
remotely  pertinent  to  the  investigation  but  perhaps  of  interest  to 
persons  working  in  other  functional  areas  in  which  the  generalized  ideas 
presented  are  applicable.  Also  appended  are  sample  worksheets  which 
might  be  used  in  situations  similar  to  the  example  presented. 
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/  ' 

II.  Decision ‘Theory  Review 

Since  man  was  created,  he  has  been  making  decisions.  As  the 
species  expanded,  other  men  (and  ofttimes  women)  started  telling  him 
how  he  should  make  (or  should  have  made)  decisions.  In  recent  years, 
startling  new  insights  have  evolved  in  the  recommended  processes  for 
decision-making.  Today  decision  theory  is  considered  almost  a  distinct 
scientific  discipline.  Does  this  mean  that,  because  v/e  all  make 
decisions  (or  suffer  the  consequences  from  them),  we  should  rush  out 
and  obtain  a  rash  of  textbooks  and  read  up  on  the  subject?  The  answer 
to  this  question  depends  on  the  nature  of  decisions  involved.  .Vost 
decision  situations  are  trivial  and  require  very  little  thought; 
therefore,  to  perform  extensive  analyses  before  making  simple  choices 
would  be  ridiculous  to  say  the  least. 

Nevertheless,  there  are  occasions  when  there  is  much  at  stake  and 
thorough  examination  and  evaluation  of  various  options  and  their 
implications  are  definitely  in  order.  In  these  important  cases,  some 
knowledge  of  the  precepts  and  principles  of  decision  theory  can  be 
extremely  helpful.  However,  it  is  stressed  from  the  outset  that  the 
study  and  application  of  decision  theory  will  not  add  to  the  information 
available  to  the  decision  maker,  but  will  merely  help  him  organize  the 
relevant  facts  and  opinions  in  a  manner  which  will  assist  him  in 
making  a  rational  choice.  Decision  theory  is  essentially  a  logically 
consistent  systematic  approach  to  the  selection  of  one  of  several 
alternative  courses  of  action  available  to  a  decision  tinker. 
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The  methodical  approach  .which  is  outUjied  in  this  section  should 
be  recognized  as  a  basic  conceptual  framework  for  the  analytical 
treatment  of  decision  situations.  It  should  only  be  used  as  a  guide 
and  not  be  considered  as  a  dogmatic  delineation  of  how  to  treat  all 
problems.  Each  decision  situation  is  unique  and  must  be  handled  in 
light  of  its  particular  characteristics.  A  primary  principle  in 
decision  theory  is  that  selection  problems  can  be  broken  down  into 
their  constituent  elements  which  are  usually  easier  to  analyze.  The 
methodology  also  puts  these  individual  elements  in  proper  perspective 
with  each  other  so,  although  they  can  be  treated  separately,  the  "big 
picture”  is  not  lost.  The  detailed  analysis  of  these  interrelated 
portions  of  the  problem  are  accomplished  in  various  ways  but  each  has 
a  common  prerequisite  _  the  willingness  to  think  quantitatively. 

This  requirement  is  established  so  the  powerful  mathematical 
techniques  associated  with  decision  theory  can  be  employed  to 
formulate  and  solve  decision  models.  No  longer  does  the  decision 
maker  have  to  rely  solely  on  emotion  and  intuition.  However,  the  use 
of  analytical  tools  by  no  means  eliminates  or  diminishes  the  importance 
of  a  decision  maker's  experience  and  his  instincts  and  insights  derived 
from  similar  situations.  Rather  than  reducing  the  need  for  human 
judgement,  the  framework  actually  provides  a  mechanism  for  the  explicit 
consideration  of  personal  experience  and  opinion.  As  a  matter  of  fact, 
the  additional  clarity  resulting  from  the  distinct  differentiation 
between  judgements  involving  the  likelihood  of  an  event  and  those 
concerning  the  worth  of  an  alternative  is  a  definite  advantage  of  the 
process. 
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There  are  really  several. processes  associated  with  the  general 

*  ‘  ... 

study  of  decision  theory.  There  is  no  agreed  standard  procedure  on 
how  to  structure  and  solve  decision  problems.  The  synopsis  which 
follows  is  a  presentation  of  a  process  that  is  typical  to  the  solution 
of  decision  problems  dealing  with  uncertainty.  The  follovang  parts, 
principles  and  procedures  are  considered  of  the  most  value  for  the 
decision  maker  who  works  primarily  in  the  reliability  and  maintain¬ 
ability  arena. 

Basic  Elements 

As  previously  mentioned,  decision  situations  can  be  broken  down 
or  decomposed  into  -acts,  .of'  factors  which  can  be  analyzed  individually 
at  first  and  collectively  later.  Of  course  it  is  more  difficult  to 
disect  and  recombine  partial  analyses  of  problems  which  are  more 
complex  or  have  greater  uncertainty  surrounding  them.  Although  the 
decomposition  effort  may  be  minimal  or  extensive,  there  are  several 
sets  of  factors  that  are  common  to  all  decisions  made  under  uncertainty. 
Depending  on  the  preliminary  results  of  a  partial  analyses,  this  common 
or  primary  group  may  be  augmented  by  an  additional  or  supplementary 
group. 

Primary  Sets.  The  primary  group  of  elements  consists  of:  a  set 
of  the  available  alternative  courses  of  action;  a  set  of  the  possible 
states  of  nature;  and  a  set  of  consequences  of  each  alternative  and 
state  of  nature  combination. 

The  action  set  will  be  denoted  by  A, 
where  A  =  ‘{aj.}  for  i  =  l,2,...,m 


6 


GRE/MAT  H/ 66-11 


and  =  a  specific  -course-of  action 

-  "  ...  "  , 

m  =  the  total  number  of  alternatives  which 
are  feasible  arid  practical 

There  is  no  prescribed  procedure  on  how  to  develop  a  list  of  possible 

actions  that  might  be  available.  Obviously,  the  compilation  of  this 

list  deserves  considerable  thought  to  assure  all  reasonable  options 

are  included.  The  decision  maker  is  encouraged  to  apply  his  ingenuity, 

imagination,  and  initiative  to  the  fullest  extent.  The  methodology 

does  require  that  members  of  the  action  set  be  mutually  exclusive  and 

collectively  exhaustive.  The  mutually  exclusive  property  limits  the 

selection  to  only  one  member  from  the  set  -  combinations  are  not 

permitted.  The  collectively  exhaustive  property  merely  means  that  the 

list  should  be  complete  in  the  sense  that  one  of  the  members  must  be 

chosen.  Thus,  the  solution  to  the  decision  situation  is  the  selection 

of  a  single  item  from  this  action  set.  The  primary  difficulty  in 

making  this  choice  is  usually  due  to  the  uncertainties  of  the  situation, 

in  other  words,  not  knowing  exactly  what  will  happen  should  a  particular 

alternative  be  selected.  These  uncertainties  usually  stem  from  unknown 

states  of  nature  which  constitutei  the  second  primary  set.  The  nature 

state  set  will  be  denoted  by  0 , 

where  0  =  {0^}  for  j  =  l,2,...,n 

and  0.  =  a  possible  event  that  can  occur  or  happen 
J  which  is  relevant  to  one  or  more  of  the 
actions,  aj, ,  under  consideration.  Nature 
exists  in  exactly  one  of  these  unknown  states. 

n  =  the  total  number  of  states  that  have  a 
potential  impact  on  the  problem. 
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As  in  the  action  set,  there-  is  -n‘6‘"sugges ted'  way  to  enumerate  all  the 
uncertainties  involved  in  a  particular  decision  situation.  Again, 
the  skills  of  the  individual  decision  maker  play  an  important  role  in 
judging  the  relevance  of  the  many  unknown  factors  bearing  on  a 
particular  problem.  The  decision  maker  usually  has  considerable 
latitude  in  assigning  members  to  the  nature  state  set.  However,  he 
must  obey  the  exclusive  and  exhaustive  rules  previously  mentioned. 
Therefore,  he  should  clearly  and  concisely  describe  each  event  in  the 
set  in  a  manner  .which  prevents  overlapping  and  assures  completeness. 
Difficulties. encountered  in  following  the  exclusive  rule  can  be 
alleviated  by  judicious  definition  and  careful  grouping  of  each  member. 
In  many  situations,  it  may  be  necessary  to  construct  the  set  by  taking 
pairs,  triplets  and  higher  order  combinations  from  other  lists  of 
more  ob'’nus  factors.  This  technique  is  also  sometimes  helpful  in 
adhering  to  the  exhaustive  rule.  Of  course,  it  is  highly  unlikely 
that  each  possible  uncertain  event  can  be  identified  in  each  and 
every  case.  The  time  and  effort  devoted  to  the  composition  of  the 
nature  state  should  be  tailored  to  the  importance  of  the  decision 
required.  Essentially,  the  practical  significance  of  the  exhaustive 
rule  is  that  the  list  of  uncertain  events  should  cover  all  the  known 
contingencies  likely  to  affect  the  selection  of  a  particular  course 
of  action.  This  is  not  to  say  that  there  might  not  be  unknown  factors 
that  bear  on  the  decision  problem.  Thus,  there  will  be  a  few  cases 
when  the  consequence  of  a  particular  action  will  depend  not  only  on 
factors  considered  by  the  decision  maker  but  also  unknown  unknowns. 
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However,  in  general,  consequences  .can  be  foreseen  and  it  inconvenient 
to  designate  them  as  a  set. 

The  third  and  final  primary  set,  the  consequent  set,  will  be 
denoted  by  C, 

where  C  =  |Cij}  for  i  =  1,2,..., m 

3  1,2, ...,n 

and  Cij  =  the  result  of  a  particular  available  course 
of  action,  a^,  if  a  specific  anticipated 
state  of  nature, flj,  occurs. 

A  consequence  is  essentially  the  position,  posture,  predicament  or 
state  of  affairs  associated  with  an  individual  optional  act  and 
uncertain  event  combination.  Thus,  conceptually,  there  are  m*n 
consequences  possible  in  every  decision  situation.  A  particular  course 
of  action,  if  selected,,  could  lead  to  one  of  n  consequences  depending 
on  which  state  of  nature,  exists.  Within  this  group,  there  are 
consequences  which  are  good,  desirable,  beneficial,  or  profitable 
as  well  as  those  which  are  bad,  unwanted,  detrimental  or  costly.  The 
final  choice  should  represent  an  optimum  balance  among  these  positive 
and  negative  features.  Sometimes  this  choice  is  obvious.  Many  times 
the  selection  requires  no  more  than  just  considerable  deliberation  of 
the  facts  on  hand.  But  there  are  some  instances  when  assessed  risks 
are  too  great  and  additional  data  must  be  obtained  and  analyzed  before 
an  effective  discrimination  among  the  alternatives  can  be  made.  When 
these  situations  occur  it  is  necessary  to  extend  the  basic  approach 
to  include  the  formulation  and  evaluation  of  supplementary  sets. 

'Supplementary  Sets.  The  supplementary  group  consists  of  two  sets  - 
one  which  includes  the  various  ways  to  obtain  additional  information 
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and  one  which  includes  the  supplementary  data  accumulated. 

m  •  »• 

The  first  supplementary  set  is  really  a  family  of  possible 
experiments.  This  set  will  be  noted  by  E, 
where  E  =  for  k  =  l,2,...,r 
and  ejf  =  a  specific  experiment 

r  =  the  total  number  of  experiments  which  are 
applicable  and  appropriate. 

Members  of  this  set  consist  of  any  data  collection  methods,  techniques, 
plans,  etc.,  which  can  be  used  to  discover  more  about  the  true  state 
of  nature.  A  working  knowledge  of  statistical  theory  is  a  definite 
asset  in  the  formulation  and  application  of  this  set.  The  manner  and 
extent  to  which  sampling  is  accomplished  directly  affect  the  validity 
and  creditability  of  the  results.  Usually,  both  the  type  of  additional 
information  needed  and  the  methods  of  acquisition  are  obvious.  The 
need  for  the  data  collected  to  be  both  representative  and  sufficient 
is  intuitive  to  most  people.  However,  if  experimentation  is  to  be  ar. 
important  aspect  of  a  complex  decision  problem,  then  the  advice  of  a 
competent  statistician  is  almost  mandatory.  His  role  will  be  to  assure 
that  the  experiments  being  considered  car.  generate  useful  outcomes. 
Whether  personally  observed  or  generated  by  a  sophisticated  experiment 
under  someone  else's  direction,  the  resultant  sample  data  constitutes 
the  other  supplementary  set. 

The  outcome  set  will  be  denoted  by  X, 
where  X  =  {x^}  for  1  =  l,2,...,s 
and  xq  =  a  possible  outcome  of  the  experiment  set  E 

s  =  the  total  number  of  potential  observations 
of  all  ejr  in  E. 
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The  size  of  this  3et  can  vary  from,  a  few  denumerable  elements  for 

*  *  ..  —  - 

simple  decision  situations  involving  discrete  variables  to  an  infinite 
list  of  items  for  complex  problems  involving  continuous  variables. 

In  most  cases,  the  enumeration  of  all  possible  outcomes  would  probably 
require  an  inordinate  amount  of  time  and  effort.  Generally,  the 
primary  interest  is  to  obtain  an  appreciation  for  the  range  of  outcomes. 
For  routine,  uncomplicated  decision  situations,  the  -lowest  arid  highest 
values  anticipated  for  the  results  of  a  small  number  of  experiments 
are  usually  either  apparent  or  easily  obtained.  However,  In  more 
complex  situations,  the  assistance  of  a  statistician  is  highly  desirable 
Both  the  range  and  other  meaningful  characteristics  of  the  expected 
results  can  usually  be  determined  from  mathematical  equations,  tables, 
charts,  and  other  statistical  tools  not  understood  by  all  decision 
makers.  One  such  tool  is  Bayes  Theorem  which  allows  the  pooling  of 
all  available  information  when  making  inferences  about  unknown  quantity 
(i.e.,  the  true  state  of  nature  0).  This  Theorem,  and  the  related 
prior  distribution  (which  reflects  knowledge  before  experimentation) 
will  be  addressed  in  more  detail  in  the  next  two  sections.  The 
importance  of  the  roles  that  these  topics  have  in  the  overall  decision 
process  will  become  apparent  shortly. 

Fundamental  Concepts 

In  the  preceding  discussion,  a  general  description  of  the  essen¬ 
tial  and  auxiliary  ingredients  was  provided  along  with  some  of  the 
conditions  and  constraints  for  their  formulation.,  Now,  general  guide¬ 
lines  will  be  outlined  on  how  to  shape  these  basic  building  blocks  and 


construct  a  fundamental  framework  within  which  a  variety  of  decision 


problems  can  be \ solved. 
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Definition  of  the  Problem'.  First  and  foremost  in  any  problem 
solution  process  should  be  a  careful  examination  of  the  issue  or  question 
raised.  Odiome  defines  a  problem  as  "the  difference  between  present 
condition  and  .desired  consition"  (Ref  154:15) .  He  also  states  that 
commitment  is  necessary  for  problem  solving  for  "the  committed  man  has 
to  choose  and  decide  among  alternate  solutions  and  moves.  The  un¬ 
committed  man  can  delay,  put  it  off,  and  not  get  things  done." 

(Ref  154:16)'.  These  "things  to  do"  constitute  the  desired  condition 
or  objectives.  Objectives  should  be  stated  in  clear  concise  terms 
which  collectively  can  be  a  guiding  light  providing  illuminating 
direction  toward  their  achievement.  The  precise  and  complete  delinea¬ 
tion  of  the  difference  between  what  is  wanted  and  what  is  available 
is  a  primary  prerequisite  for  future  efforts  to  bridge  the  gap  by 
enumerating  ar.d  evaluating  the  alternate  avenues  available. 

Assessment  of  the  Uncertainties.  The- Impact  of  each  of  these 
options  is  dependent  on  an  unknown  state  of  nature  existing  at  the 
time  action  is  taken.  Although  the  exact  state  is  unknown,  it  is 
assumed  to  be  a  member  of  the  set,  Q  ,  of  possible  states.  In  most 
situations,  the  members  of are  not  all  equally  likely  to  occur. 

In  fact,  the  decision  maker  may  have  encountered  similar  situations 
and  thus  has  a  basis  for  weighing  a  particular  6-  more  heavily-  than 
others . 

The  English  language  has  several  words  to  describe 
aspects  of  the  uncertainty  that  is  felt  on  such  occasions: 
one  of  them,  likely,  has  been  used  in  the  previous  sentence. 

Others  are  nrobable.  credible,  plausible  ar.d  expressions 
derived  from  words  like  chance  or  odds.  Our  aim  is  to 
describe  this  uncertainty  numerically;  for  number  is  the 
essence  of  the  scientific  method  and  it  is  by  measuring 
things  that  we  know  them.  (Ref  137:13). 
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The  tern  which  will  ba  used-  throughout  this  thesis,  is  probability. 

It  is  intended  that  a  generic  definition  apply  to  this  yardstick  for 
measuring  uncertainty.  Subsequent  to  the  following  discussion  bn  the 
various  kinds  of  probability,  the  word  will  be  used  without  qualifying 
adjectives. 

The  word  '’probability"  is  used  by  practicing  mathematicians  and 

statisticians  in  several  different  ways  and  means  different  things  to 

different  people.  To  circumvent  these  semantic  obstacles,  various 

descriptors  have  been  used  to  provide;  more  explicit  definitions. 

“typical  of  the  phrases  often  used  are  those  discussed  by  Good: 

...a  physical  probability  (also  called  "material  probability," 
"intrinsic  probability,"  "propensity,"  or  "chance")  is  a 
probability  that  is  regarded  as  an  intrinsic  property  of  the 
material  world,  existing  irrespective  of  minds  and  logic... 

A  psychological  probabi  li-ty  is  a  degree  of  belief  or 
intensity  of  conviction  that  is  used  for  betting  purposes, 
for  making  decisions,  or  for  any  other  purpose,  not  neces¬ 
sarily  after  mature  consideration  and  not  necessarily  with 
any  attempt  at  "consistency"  with  one's  other  opinions... 

When  a  person  or  persons,  called  "you,"  uses  a  fairly  consistent 
set  of  probabilities,  they  are  called  subjective  ("personal") 
or  multi  subjective  ("multi  personal")  probabilities.  (Ref  94:6). 

Thus,  physical  probability  corresponds  to  the  relative  frequency 

interpretation,  and  is  measured  by  observing  how  often  a  particular 

event  occurs  in  relation  to  the  total  number  of  attempts  made.  Subjective 

probability  (the  special  case  of  psychological  probabilities  to  which 

future  discussion  will  be  limited)  applies  when  "one  is  quantifying 

his  personal/judgements  based  on  his  experience  and  knowledge,  insight 

and  information."  (Ref  116:  28).  The  mathematical  properties  of  both 

these  general  classes  must  obey  the  postulates  and  laws  of  probability 

theory.  Using  the  event  set  notation  and  defining  P(  )  as  "the 

probability  of  the  event  (  ) ,  the  throe  basic  requirements  for 
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mutually  exclusive  '(or  disjoint)  ^events  are  as  follows: 

(1)  o<p  (®p<  1  '  • 

(2)  p(e)  =  i 

(3a)  P(0^  or  $2  or  •••)  =  P(®i)  +  +  ••• 

(3b) . ?($ ,  and  any  other  6.)  =  C 
J  J 

Rule  (1)  represents  the  only  new  requirement  since  rules  (2)  and:  (3a, 

3b)  are  merely  reexpressions  of  the  exhaustive  and  exclusive  rules, 
respectively.  Rule  (l)  can  also  be  satisfied  if  odds  are  quoted 
for  uncertainty.  For  example,  if  a,  particular  event  is  favored  4  to  1, 
the  equivalent  statement  is  that  it  is  likely  to  occur  4  out  of  5 
times  or  with  .80  probability.  Another  aid  in  establishing  a  number 
between  0  and  1  as  a  measure'  of  likelihood  is  the  comparison  of  the 
chances  of  a  particular  event  with  those  of  a  random  point  falling 
within  a  designated  area  of  a  unit  square.  (Ref  137:19). 

Comparison  of  Consequences.  In  addition  to  evaluating  the  data 

(either  objective  or  subjective)  relating  to  the  likelihood  of  the  state 

of  nature,  P(0.),  the  decision  maker  must  also  examine  the  potential 
J 

impacts  of  the  individual  choices,  .  As  previously  stated,  not  all 
consequences  are  equally  favorable.  Thus,  the  decision  maker  should 
list  the  possible  consequences  in  a  relative  ranking  sequence  which 
reflects  the  degrees  of  achievement  toward  the  objectives.  Obviously, 
the  particular  consequence  which  represents  the  greatest  step  toward 
the  goal  is  to  be  the  most  preferred.  Also,  great  gains  should  rank 
higher  than  shorter  ones.  Therefore,  as  Lindley  explains: 
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■It  follows  that  the  -next  task  is  to  provide  something  more 
than  just  a  ranking,  of- the  consequences.  In  order  to  do 
this  a  standard  is  introduced  and. coherent  comparison  with 
it  provides  a  numerical  assessment,  just  as  with  the  uncertain 
events.  In  the  case  of  probability.,  the  standard  was  a 
random  point  in  a  unit  square.  For  the  consequences,  two 
reference  consequences  are  used;  one  of  these  is  better  than, 
or  at  rate  not  worse  than,  any  of  the  consequences  in  the  » 

relevant  table;  the  other  is  similarly  worse  than,  or  at 
most  not  better  than,  all  the  C*j.  (Ref  137:52). 

He  goes  on  to  develop  a  separate  probability  measure  relating  to  the 

attractiveness  of  a  particular  C^j  in  comparison  with  the  most 

preferred  consequence.  He  calls  this  numerical  measure  "a  utility 

of  the  consequence."  (Ref  137:53). 

Characteristics  of  Utility.  The  numerical  measure  for  consequences 

does  not  have  to  be  a  probability  function.  In  fact,  a  quantitative 

yardstick,  although  highly  desirable,  is  not  mandatory.  Qualitative 

expressions  may  be  used  if  more  appropriate.  Killer  states  that  "in 

addition  to  the  common  practice  of  measuring  the  utility  of  consequences 

on  an  objective  scale  such  as  dollars,  gain  may  be  a  more  subjective 

concept  including  factors  such  as  reputation,  happiness,  security,  or 

any  other  characteristic  associated  with  well-being."  (Ref  14S:3~o). 

In  addition,  Schaifer  indicates  that  "consequences  might  also  involve 

"physical  assets,  technological  know-how, ...  the  .behavior  of  various 

people, .. .and  the  decision  maker’s  own  personal  position..." 

(Ref  186:40).  The  point  really  is  that  there  is  no  common  denominator 

or  standard  dimension  against  which  the  worth  of  a  consequence  can  be 

gauged.  However,  there  is  a  definite  advantage  to  using  numerical 

measures,  because  they  are  easier  to  handle  mathematically.  The  basic 

requirement  in  assigning  a  quantitative  index  to  a  value  judgement  is 
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coherence.  A  decision  maker  is. .considered  coherent  if  he  uses  a 
utility  function  vhich  assigns  a  higher  utility  number  to  a  most 
preferred  consequence  and  equal  numbers  to  consequences  for  which  he 
is  indifferent.  A  particular  quantity  for  a  utility  measure  may  be 
designated  U(Cj^)  and  may  fall  within  any  arbitrarily  selected  range 
that  is  appropriate  to  the  situation. 

Calculation  of  Expected  Value.  Once  the  numerical  measures 
P(flj)  and  U(C^j)  have  been  determined,  the  next  step  is  to  cpmbine 
them  in  a  manner  that  the  relative  merits  of  the' various  a^  can  be 
assessed.  To  do  this,  the  quantity  known  as  expected  value  will  be 
used.  The  expected  value,  or  mathematical  expectation,  of  a  particular 
a^,  written  E(a^)  can  be  computed  from  the  following  relationship. 

E(ai)  =  £  U{C  )P(fl  ) 

H  J  (2-1) 

Thus,  E(ai)  is  a  measure  of  the  extent  that  a^  can  solve  the  problem 
considering  the  circumstances  known  to  the  decision  maker.  The 
logical  choice  is  to  select  the  alternative  from  A  which  has  the 
greatest  E(a^) .  Since  this  choice  is  highly  dependent  on  the  P(0j) , 
these  quantities  must  reflect  all  known  information  about  ©  -  not 
necessarily  just  the  recorded  results  of  sampling.  Bayes  Theorem 
serves  as  the  basis  for  the  current  practice  of  combining  subjective 
and  test  data. 
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"III.  Bayes  Theorem 

Bayes  Theorem  is  essentially  &  simple  relation  between  probabilities 
that  two  different  events  will  occur.  The  basic  expression  which 
describes  the  relationship  is 
P(A/B)  =  P(A)P(B/A) 

P(bJ  (3-D 

where  the  /  is  to  be  read  "given." 

Development 

There  is  really  nothing  special  about  the  formula  per  se  -  its 
derivation  is  quite  straightforward . 

Consider  the  "probability  diagram"  depicted  in  Figure  1,  where  A 
and  B  represent  two  events  (not  disjoint  as  were  the  ) . 


Probability  Diagram 
(from  Ref  213*.  344) 

The  probability  of  event  B  given  that  event  A  has  occurred,  P(8/A)  is 
the  portion  that  common  shaded  area,  P(AB) ,  is  to  the  total  area  P(A). 
In  equation  form, 

P(B/A)  =  P(AB) 

pTa)  (3-2) 
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Likewise,  P(A/B)  can  be  determined  bgr 
P(A/B)  =  P(AB) 


(3-3) 


lot*  P(AB)  ia  conon  to  both  aquations  and  thus 
P(AB)  =  P(A/B)  P(B)  =  P(B/A)  P(A) 

Vhen  aaeh  tar*  of  tha  latter  equality  is  divided  by  P(B) Bayas 
Theorem  results. 

P(A/B)  =  F(A). 

Interpretation 


(3-4) 


So  what  Is  all  tha  fuss  about  an  easy  rearrangement  of  terms? 

Tha  answer  is  in  how  these  terms  may  be  interpreted.  Addressing  each 
tarn  individually  and  replacing  A  with  tj:  ;  and  B  with  xjr 

p(*j  )  =  Prior  probability  (what  is  known  about  ^ 

before  additional  data  is  available) 
P(x^/$j)  =  Likelihood  of  observing  x^  (how  probable 

is  the  sampling  results,  assuming  the  true 
state  of  nature  is  a  particular  dj)  v 

P(xj)  =  Probability  of  test  observations  for  any  0j 


P(*j/x) 


in  0 


=  Posterior  probability  (what  is  known  about  6 j 
after  has  been  obtained  and  analyzed) 


Thus,  Bayes  Theorem  provides  the  framework  which  allows  redetermination 
of  P(0j )  based  on  additional  information.  Decisions'  dependent  on 
knowledge  of  P(0j )  are  better  when  based  on  all  available  information. 
But  thore  is  some  controversy  between  the  Bayes  approach  and  classical 
statistical  methods  when  some  or  all  of  the  prior  knowledge  consists 
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of  theoretical  considerations,  design  analyses,  engineering  judgement, 
•to.  '  — 

Tha  key  to  the  question  concerning  the  applicability  of  Bayes 
Theorem  for  updating  P(#j)  is  the  use  of  subjective  rather  than 
objective  information  to  establish  the  prior  probability.  The  argument 
advanced  by  the  purists  is  that  the  introduction  of  intuition  and  guess¬ 
work  into  Statistics  constitutes  an  unnecessary  bias  and  the  resultant 
inferences  made  are  not  valid.  The  reason  for  these  doubts  is  centered 
around  the  various  definitions  of  probability  discussed  in  the  .previous 
section.  Essentially,  there  are  two  schools  of  thought  which  are 
prevalent  today.  For  future  discussions,  the  terms  designated  by  k’eir 
for  the  advocates  of  these  two  philosophies  -will  be  used: 

(1)  "Classicist"  which  will  refer  to  the  group  which 
adopts  the  frequency  interpretation  of  probability.  This 
faction  believes  in  the  concept  of  unknown  parameters 
located  at  one  unknown  point,  which  can  be  estimated  with 
increasing  precision  as  test  data  builds  up;  information 
of  a  non  frequency  nature  cannot  be  directly  used  ir.  this 
estimate.  Probabilities  based  upon  outcomes  of  games  of 
chance  (e.g.,  flipping  coins,  tossing  dice,  dealing  cards) 
are  of  course  subject  to  a  frequency  interpretation.  The 
ratio  of  successes  to  total  trials  over  the  very  long  term 
can  be  nearly  equated  to  the  probability  of  the  event, 

(2)  "Subjectivist"  which  will  refer  to  the  group  which 
while  dealing  with  the  frequency  application  of  probability 
to  games-  of  chance  ar.d  ot-.ier  applicable  situations,  also 
believes  that  probability  theory  can  bo  applied  to  questions 
of  degree  of  belief  in  propositions  (e.g.,  the  probability 
that  there  is  life  on  Kars).  This  faction  believes  it  is 
sensible  to  talk  of  probability  distributions  of  parameters 
based  upon  degree  of  belief  in  the  location  of  the  parameter; 
this  .permits  both  frequency  and  non  frequency  information 

to  be  combined  using  the  central  framework  of  Bayes  Theorem. 

(Ref  213:345). 
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It  is  hot  the  intent  of  this  work  to  delve  into  the  controversy 
but  merely  point  but  its  existence.  A  rather  comprehensive  discussion 
of  the  pros  and  cons  of  each  of  the  approaches  is  presented  by  Hahn, 
in  the  proceedings  of  the  1965  General  Electric  Seminar  (Ref  214: Sec  VIII) ... 

In  spite  of  the  controversy,  Bayes  Theorem  and  the  more  inclusive 
field  of  what  is  sometimes  referred  to  as  "Bayesian  Statistics"  have 
been  described  as  an  important  step  forward  in  removing  some  of  the 
constraints  of  classical  theory  (Ref  173) .  Some  authors  point  out  that 
"Bayesian  techniques  complement  classical,  statistical  methods  rather 
than  replacing  them."  (Ref  74:5)-.  This  is  especially  true  when  the 
techniques  are  applied  to  reliability  estimation  and  assessment  problems. 
Application  to- 'Reliability  Problems 

Schulho?  and  Lindstrom  have  stated  three  primary  factors  which 
have  generated  the  need  to  replace  time  honored  classical  methods  with 
more  modem  schemes: 

1,  Products  are  becoming  more  complex  and  correspondingly 
expensive. 

2,  Time  is  at  a  great  premium. 

,3.  High  reliability  is  being  both  demanded  and  achieved. 

(Ref  187:634). 

The  paradox  is  one  of  greater  requirements  but  fewer  resources  to 
verify  achievement.  The  "Classicist"  by  ignoring  prior  subjective 
estimates  on  the  range  of  the  failure  rate  of  an  item  in  essence  makes 
the  implicit  assumption  that  the  item  nay  possess  any  failure  rate 
(e.g.,  0<  A<»)  (Ref  153:3).  This  hypothesis  is  extremely  costly 
from  the  standpoint  of  the  inordinate  amount  of  testing  required  to 
narrow  in  on  the  true  value  for  a  reasonable  degree  of  assurance. 
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The  Bayesian  approach  recognizes  the  value  of  past  experience  and 
admits  any  data  -  theoretical  limits,  quantified  beliefs,  intuitions, 
whatever  -  into  the  analysis.  This  prior  information  reduces  the  scope 
of  sampling  to  a  finite  range  with  resultant  savings  in  time  ar.d  money. 
Iterative  analyses  may  be  performed  to  provide  periodic  updates  con¬ 
cerning  a  system’ s  -reliability. 

Bayesian  methods  can  use  knowledge  gained  frcm  development 
testing  to  indicate  the  reliability  of  equipment  at  each 
stage  of  its  development.  Tests  may  also  be  conducted 
during  design  development  for  purposes  other  than  reliability 
-estimation  with  full  expectation  that  the  data  can  be  inte¬ 
grated  into  a  reliability  estimation  procedure  with 
reasonable  statistical  validity-;  It  is  possible  to  ascertain 
by  these  methods  whether  a  specified  system  reliability 
requirement  has  a  reasonably  good  chance  of  being  achieved. 
Conversely,  these  methods  will  show  whether  special  effort, 
such-  at  redesign  or  modification,  is  warranted,  by  indicating 
the  existence  of  a  low  .probability  of  design  reliability 
achievement. 

By  having  this  objective  information  available  concurrently 
with  each  phase  in  design  development,  the  designer  is 
enabled  to  make  necessary  improvements  at  an  earlier  state. 

This  makes  revisions  more  compatible  with  costs,  schedules, 
tool  design,  and  other  important  factors,  and  makes  achieve¬ 
ment  of  the  contractual  reliability  requirement  more  certain. 

As  a  consequence,  the  number  of  tests  required  to  achieve 
and  demonstrate  a  reliable  design  can  be  reduced,  resulting 
in  time  and  cost  savings.  (Ref  74:5). 

In  summary,  the  following  are  considered  practical  advantages  of 
using  Bayes  techniques  for  estimating  reliability: 

1.  The  estimates  take  both  predictions  and  test  data  into 
account. 

2.  The  estimates  give  reasonable  results  for  little  or  no 
test  data. 

3..  The  estimates  agree  with  the  classical  estimates  for 
large  samples.  (Ref  187:634). 
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.  Of  course,  there  are- drawbacks  to  ueing  Bayesian  techniques,  in 
addition  to  the  issues  raised  by  the  "Classicists,"  there  is  the 
obvious  problem  of  erroneous  initial  information.  What  happens  if 
incorrect  assumptions  are  made  or  faulty  logic  was  used  to  formulate 
a  considered  opinion?  How  accurate  car.  an  overall  final  estimate  or 
assessment  be  if  based  on  imprecise  inputs?  Fortunately,  Bayesian 
methods  have  an  inherent  corrective  feature  since  as  the  quantity  of 
subsequent  test  data  increases,  the  initial  estimates  become  over¬ 
shadowed.  If  desired,  the  effect  of  incorrect  prior  information  can 
be  reduced  to  any  stated  level  by  sufficient  additional  testing.  As 
previously  stated,  'Bayesian  estimates  agree  with  "Classicist"  estimates 
after  an  extensive  amount  of  test  and  operational  life  data  has  been 
obtained. 

The  question  which  must  be  answered  now  is  "How  does  one  quantify 
his  subjective  appraisal  of  an  item’s  reliability  in  a  manner  that  is 
usable?"  This  is  accomplished  by  using  available  reliability  pre¬ 
dictions  and  related  statements  of  uncertainty  surrounding  them. 
Basically,  the  parameters  of  the  applicable  system  failure  density 
function  are  considered  to  be  random  variables.  A.  probability 
distribution  function  for  a  parameter  is  established  based  on  the 
point  estimate  and  expected  range  of  the  predicted  value.  This  para¬ 
meter  distribution  is  known  as  the  prior  distribution  and  is  subject 
to  modification  as  test  data  becomes  available.  Typical  prior 
distributions  encountered  in  reliability  and  instructions  on  how  to 
generate  them  are  the  subject  of  the  next  section. 
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IV.  Trior  Distributions 

The  heart  of  a  Bayesian  statistical  inference  is  that  a 
probability  distribution  is  assumed  to  exist  for  the  unknown  true 
state  of  nature  9  .  This  prior  distribution  essentially  reflects  the 
amount  of  knowledge,  or  degree  of  belief,  before  the  results  of  expe¬ 
rimentation  are  available.  If  absolutely  no  information  is  known, 
all  values  of  9  are  equally  likely,  and  logic  dictates  a  Uniform 
prior  distribution.  However,  in  virtually  all  reliability  problems, 
there  exists  a  significant  amount  of  information  from  generic  or 
similar  parts  documented  in  various  handbooks.  Also,  according  to 
Gottfried  and  Weiss,  "experience,  indicates  that  the- failure  rate  of  any 
device  is  bounded  on  the  left  (non-negative)  and  skewed  to  the  right  - 
larger  failure  rates  than  expected  are  less  surprising  than  smaller 
values."  (Ref  100:603).  Although  there  is  no  widely  agreed  rules  on 
how  to  select  a  suitable  prior  distribution,  Babillis  and  Smith  have 
established  what  is  considered  to  be  minimal  criteria: 

1.  The  prior  distribution  must  adequately  reflect  what  is 
actually  known  before  test  data  become?  available.  That  is, 
the  distribution  must  be  consistent  with  the  available  prior 
knowledge  of  a  component's  reliability. 

2.  The  prior  distribution  should  not  imply  any  assumptions 
about  unknown  information  concerning  the  reliability  of  a 
component.  In  other  words  it  should  remain  as  maximally 
noncommittal  and  as  unprejudiced  as  possible  concerning 
things  which  are  ccnridered  unknown. 
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■  3.  The  prior  distribution  . should  r.ot  lead  to  absurd 
-conclusions  concerning  the  component*  s  reliability  when 
it  is  modified  by  the  data.  The  selected  apriori  distribu¬ 
tion  should  not  lead  to  results  which  are  inconsistent 
with  what  is  known  or  intuitively  felt. i. However,  it  is 
important  to  realize  that  these  inconsistencies  may  only 
be  evaluated  in  terms  of  what  was  originally  thought  to 
be  known.  If  pertinent  information,  is  withheld  in 
establishing  the  prior  distribution,  this  same  information 
may  not  be  used  to  discredit  the  resulting  posterior 
distribution. 

4.  The  resulting  equations  should  be  tractable  by 
available  mathematical  methods.  This  is  purely  a  prag¬ 
matical  consideration  based  solely  on  the  desire  for  an 
answer.  Through  this  sane  crack  in  the  otherwise  logical, 
framework  also  creeps  a  certain  element  of  empiricism 
which  requires  drawing  upon  experience  to  get  the  technique 
started^  (Ref  6:357). 

Although  the  above  criteria  are  general,  they  do  serve  the  purpose  of 
narrowing  the  selection  process.  Perhaps  someday  simple  algorithms 
for  establishing  priors  will  exist.  As  of  this  writing  there  is 
considerable  effort  underway  to  find  ’ways  and  means  for  formulating 
probability  distributions  for  reliability  indices.  For  example,  Rome 
Air  Development  Center  has  a  current  study  contract  with  IJughes 
Aircraft  Company  to  develop  methods  for  fitting  prior  distributions 
to  empirical  data  and  combining  priors  from  similar  but  not  identical 
equipment.  The  reported  results  of  initial  efforts  are  quite  promising. 
It  has  been  concluded  that  it  is  entirely  feasible  to  fit  prior 
distributions  to  equipment  level  Mean  Time  Between  Failure,  MTBFs, 
although  the  amount  of  data  currently  in  existence  to  do  so  is  some¬ 
what  limited.  It  was  also  determined  that  the  probability  of  MTBF 
is  usually  well  described  by  an  Inverted  Gamma  Distribution  when  the 
assumed  equipment  failure  time  is  exponentially  distributed.  (Ref  182). 
Since  MTBF  or  its  reciprocal,  the  failure  rate  A ,  are  the  most 
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commonly  used  measures’  for  time  ^dependent  reliability,  one  merely 

.  •  „ 

has  to  determine  which  member  of  the  family  of  Gamma  type  distributions 
is  applicable  to  the  situation.  He-:  does,  one  translate  his  knowledge 
concerning  the  most  likely  single  value  and-  range  of  uncertainty  for 
MT8F  or  X  into  a  Gamma  distribution? 

Assigning  Gamma  Parameters 

The  Gamma  family  of  probability  distributions  meets  all  the 
previously  stated  criteria.  In  the  most  common  fora,  it  is  described 
by  two  parameters  and  is  'very  flexible.  Also,  the  selected  parameters 
combine  readily  with  Poisson  sampling  statistics  (the  Poisson  process 
is  an  experiment  which  observes  f  failures  in  t  time)  which  is  a  pre¬ 
requisite  to  meet  the  fourth  criterion  for  priors.  If  a  Gamma  prior 
is  updated  with  Poisson  data  the  resultant  posterior  is  also  a  Gamma 
distribution.  This  property  is  very  convenient  for  calculation 
purposes. 

Now  that  thei  .suitability  of  a  Gamma  form  has  been  somewhat 
Justified,  an  explanation  on  how  it  fits  into  the  overall  decision 
making  procedure  is  in  order.  If  an  equipment  failure  rate  X  is 
assumed  to  be  a  random  variable,  then 

P(#)  =  gU;.,b)  *  i,‘  xa-1«-tx  ' 

TH  (4-1) 

for  a,b,X>0 

The  probability  of  a  particular  outcome  of  the  Poisson  experiment 
(x  =  f  observed  failures  in  t  time) ,  given  the  true  state  of  nature 
»  (l  =  X)  can  be  expressed  as 

P(x/X)  =  p(f/t,X)  =  (Xt)r  e‘*1 
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The  remaining  term,  P(x),  of-  the* Bayes  formula  can  be  determined  in 
the  following  manner 


P(x)  =  j  ?(x/$)?(e)d$ 

-  J*° p(f/t,X)g.(X.j,a/b)dX 


Thus,  from  the  Bayes  relation ship  (3-1) 


P(0/x)  =  P.(X/f,t)  = 


(4-3) 


(Xt)f  e~Xt  ba  Xa~1  e~bX 

.  f! _ r(a) 

(Xt)f  e'Xt  ba  Xa_1  e~bX  dX  (4-4) 


Now  that  this  specific  case  has  been  defined,  the  general  expressions 

involving  $  and  x  will  no  longer  be  used,  and  the  expression  for  the 

posterior  distribution  can  be  reduced  to 

j^a+f-1  e-  X  (b+t) 


p(W-)  =  ts - ' - 

by  letting  y  =  X  (b  +  t)  the  denominator  becomes 

J  W 

which  can  be  solved  by  rearranging  terms, 

(.  a+f  foo  a+f-1  -y  .  .a+f 

1  ]  /  y  e  dy  =  [ _JL_)  T(  a  +  f) 

b  +  t/  J0  \b  +  t/ 

Substituting  this  result  in  equation  (4-5) 

P(X/f,t)  =  (b  +  t)a+f  X^'1  e’x(b+t) 

T(a  +  f) 

=  g(X;a  +  f,  b+t) 


(4-5) 


(4-6) 


(4-7) 
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With  this  proof,  a  better  appreciation  for  the  interpretation  of 
the  meaning,  of  the  parameters  a  and  b  is  possible.  Since  a  and  f  are 
equivalent  quantities,  the  former  can  be  considered  what  has  been 
termed  as  a  "pseudo- failure" ;  likewise,  b  can  be  thought  of  as  "pseudo¬ 
time"  (Ref  214:5-6).  So  by  assigning  values  for  a  and  b,  one  may 
assert  his  extent  of  subjective  confidence  in  predicted  equipment 
failure  rates.  This  may  be  done  by  examining  the  mean  and  variance 
of  the  Gamma  form  defined  in  equation  (4-1) , 

"-6 

*2  _  a  _ 

~  b2  '  b  (4-8) 

If  the  predicted  value  Xp  is  equated  to  the  mean,  then 

Xp  s  S 
H  b 

a2  s  H 

b  (4-9) 

Thus,  by  assigning  a  and  b,  one  is  really  stating  his  belief  concerning 
the  number  of  "pseudo- failures"  and  the  amount  of  "pseudo- time" 
reflected  in  the  predicted  value.  For  example,  if  Xp  =  50 \X  1C~‘, 
then  the  following  combinations  are  possible  and  are  listed  ,in  order 
of  increasing  certainty  from  left  to  right: 


£ .  0.1.  0.5 


JL 


10 


b  *  2000  10,000  20yC0P  100,000  200,000 
The  importance  of  carefully  selecting  values  for  a  and  b  that 
realistically  correspond  to  prior  knowledge  cannot  be  overemphasized. 
Understatements  are  costly  because  of  extensive  sampling  required  to 
narrow  the  resultant  wide  range  (i.e.,  large  a2).  Conversely, 


27 


GRE/MATH/66-11 


overstatements  are  ill  advised  because  of-  .the  significant  contribution 
that  the  prior  will  have  on  the  posterior  and  thus  the  decision  to  be 
made.  With  these  cautions,,  three  methods  will  now  be  discussed  which 
address  how  to  select  values  for  a  aiid  b. 

Upper  Bound  Method.  Two.  known  bits  uf  information  must  be 
provided  if  two  unknown  quantities  (in  this  case  a  and  b)  are  to  be 
determined.  The  predicted  failure  rate  Xp  represents  only  half  of 
the  required  input.  This  value  must  be  combined  with  another  statement 
relating  to  additional  prior  knowledge  about  the  failure  rate.  One 
way  to  encode  a  conviction  of  belief  concerning  a  reliability  prediction 
is  to  assign  a  probability,  p ,  that  the  true  failure  rate  is  no  greater 
than  a  particular  upper  limit  Xy  .  Once  this  is  done,  then  Xp  can  be 
considered  to  be  either  the  mean  or  median  value  of  the  prior 
distribution  and  published  tables  or  graphs  can  be  used  to  select 
values  for  a  and  b  which  satisfy  the  probability  statement. 

(Ref  214:A2-37).  The  value  a  is  the  same  for  any  Xp  and  Ay  combina¬ 
tion  with  an  identical  discrimination  ratio  Ay  ,Ap  and  extent  of 
certainty, p  .  It  was  found  convenient  for  later  use  to  tabulate  an 
assortment  of  what  is  considered  typical  Ay/Ap  and  P  .  The  values 
for  a  for  discrimination  ratios  from  1.50  to  10  and  P  from  .67  to  .99 
are  given  for  Xp  equal  to  the  mean  in  Table  1  and  Xp  equal  the 
median  in  Table  II. 

Coefficient  of  Variation  Method.  Another  way  to  describe 
uncertainty  associated  with  a  prediction  is  to  state  the  extent  of 
possible  inaccuracy  in  terms  of  an  estimated  standard  deviation.  If 
an  engineer  states  WI  think  Xp  is  the  true  failure  rate  but  I  could 
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be  off  by  a  factor  of  m"  then  X-p-'-and  m  can  be  used  'to  compute  a  and 

.  .  ^ _  * 

b  as  follows: 


(4-10) 


(4-11) 


Also  from  equation  (4-10)  it  is  noted  that  m  is  the  ratio  of  the 
standard  deviation  to  the  mean  which  is  defined  to  be  the  coefficient 
of  variation.  To  assess  the  impact  that  the  choice  of  m  will  have  on 


the  final  decision,  the  mean  of  the  posterior  must  be  examined.  The 


revised  estimate  of  the  failure  rate  Xp  may  be  written 


b  +  f  t  f  io 

m2  Xp  (4-12) 

Thus,  if  there  is  no  uncertainty  associated  with  Xp  then  m  =  0 
reflects  total  confidence.  In  this  case  Xp  =  Xp  irrespective  of 
test  data.  Conversely,  m  =  oo  corresponds  to  total  ignorance  and 
Xp  =  f/t,  which  is  the  best  estimate  obtained  from  test  data  only. 
Babillis  and  Smith  have  stated: 


For  the  time  being  the  Gamma  prior  is  being  applied  with  either 
m  =  0.75  or  m  =  1.75  depending  on  whether  the  test  data 
reflects  no  failures  or  some  failures  respectively.  General 
experience  with  this  system  has  been  favorable,  and  develop¬ 
ment  efforts  have  been  planned  to  provide  substantiation  and 
or  refinement  of  the  approach.  (Ref  6:361). 
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Also,  Feduccia  reports  that  based  or  data. dealing  with  observed  vs 
predicted  MTBFs  of  over  one  hundred  ground  electronic  equipment  and 
systems,  the  value  of  m  was  found  equal  to  1.38.  If  similar  data 
exists  for  other  types  of  equipment,  then  a  more  meaningful  value  for 
m  can  be  determined-  from  the  sample  means  and  standard  deviation. 

Equivalent  Test  Time  Method.  The.  last  technique  to  be  discussed 
in  this  paper  addresses  the  predetermined  contribution  that  a  predicted 
value  will  have  in  the  final  decision.  The  basic  approach  reflected 
in  this  method  is  to  relate  the  uncertainty  associated  with  the 
prediction  with  the  amount  of  usage  experience  likely  to  be  encountered 
before  the  final  decision.  If  one  has  high  confidence  in  initial 
predictions,  he  has  less  need  for  additional  experimentation. 
Conversely,  if  predictions  are  considered  somewhat  inaccurate,  then 
greater  reliance  on  test  results  is  in  order.  Usually,  the  tim.-j 
associated  with  a  test  program  is  either  specified  or  can  easily  to 
estimated.  Then  any  amount  of  "pseudo- time"  can  be  assigned  to  be 
compatible  with  a  desired  extent  of  contribution  to  the  total  time 
on  which  the  final  decision  will  be  based.  For  example,  if  a  10,GC0 
hour  test  program  is  expected  and  the  decision  to  be  made  it  to  be 
based  on  90%  test  results  and  10%  predictions  then  b  =  10C0  would  be 
an  appropriate  assignment. 

If  the  preselected  portion  of  expected  test  time,  b,  is  combined 
with  Ap,  a  unique  Gamma  distribution  can  be  defined.  The  obvious 
need  is  a  relationship  between  prediction  uncertainty  and  the  extent 
that  the  prediction  should  contribute  to  the  posterior  estimate. 

There  are  no  established  guidelines  or.  this  issue,  however,  it  is 
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Suggested  that  after  careful  arid  considered  thought,  a  relationship 
could  be  devised  for  a  particular  program.  For  illustrative  purposes 
in  the  next  section,  a  hypothetical  relationship  is  given  in  Table  III 
below. 

TABLE  III 

Equivalent  Test  Time  for  Priors 


Degree  of  Belief 

Percent  Contribution 

Divisor  for 

in  Predicted  Value' 

p  QV*  pf* 

Test  Time-''?"* 

0.99 

0;75 

'V 

U-0.33 

0.90 

1.00 

0.75 

0^25 

3.00 

0.67 

.  o:i? 

5.00 

In  addition  to  time  dependent  reliability  .problems  for  which 
Gamma  distributions  apply,  there  are  cases  when  time  is  not  a  factor. 
In  these  situations,  it  has  been  shown  that  distributions  from  the 
Beta  family  meet  the  previously  stated  criteria  for  priors. 

Assigning  Beta  Parameters 

As  with  the  Gamma  family,  a  Beta  distribution  is  also  defined  by 
two  parameters  and  thus  a  wide  range  of  priors  is  possible.  The 
assigned  parameters  combine  readily  with  Binomial  sampling  statistics 
(the  Binomial  process  is  an  experiment  which  observes  s  successes 
in  n  trials).  If  a  Beta  prior  is  updated  with  Binomial  data,  the 
resultant  posterior  is  also  a  Beta  distribution.  In  this  case,  if 
the  success  ratio  p  is  assumed  to  be  a  random  variable,  then 
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,  (i-ttsp*4  (i-p)' ’♦*1  • 

0(p;>,v )  =  - - - - - 

T  (#-l)!(f  -1) !  (A-13) 

for  0  1  p  5  1 
♦,  *  >  0 

The  probability  of  a  particular  outcome  of  the  Binomial  experiment 
given  p,  can  be  expressed  as  follows 


b(./n,p)  = 

st  (n-s)t 

It  can  be  shown  that  the  posterior  distribution,  P(p/s,n),  is  also 
Beta,  fi(p;4  -r  s,  n  +  n) .  The  mean' and  variance  of  the  two  distribution 
are: 


Prior 


Posterior 


P  = 


P  * 


♦  .+  s 
i|  +  n 


ff2_  4(*J  -  $2  2_  (  $  +  s)  ( if  -<(>+  r.-s) 

v  -1  )  (  V  +  n)  (  n  +  n-1) 

The  same  rationale  used  to  assign  a  and  b  for  a  prior  Gamma  distribu¬ 
tion  can  be  used  to  select  $  and  v  for  a  prior  Beta  distribution.  This 
discussion  on  the  Beta  distribution  is  presented  for  comparison  with 
the  time  dependent  situation  for  completeness  only.  The  example  in 
the  next  section  does  not  illustrate  the  practical  application  of  this 
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V.  Bayesian  Reliability  Assessment  Example 

Now  that  the  necessary  building  blocks  have  been  explained,  they 
will  be  used  to  construct  a  framework  for  the  periodic  assessment  of 
system,  reliability  based  on  analytical  predictions  and  results  of  only 
limited  testing.  The  proposed  scheme  will  be  presented  by  way  of  a 
hypothetical  example. 

The  Original  Low-level  Detector,  OLD,  system  has  been  in  service 

for  the  past  twenty  years.  Because  of  break  throughs  in  technology 

pertaining  to  electronic  jamming  devices,  this  system  is  now  only 

marginally  effective  in  tracking  targets.  Also,  the  OLD  design  is  of 

vacuum  tube  vintage  and  has  required  increasing  amounts  of  downtime  for 

maintenance  in  recent  years.  Logistics  costs  to  support  this  system 

are  inordinate  because  of  limited,  sources  of  supply,  since  most  tube 

manufacturers  now  concentrate  on  the  production  of  solid  state  devices. 

A  modern  more  effective  system  is  currently  being  developed  to  replace 
# 

the  obsolete  OID  system. 

The  Network  for  Early  Warning,  NEW,  system  prototype  has  been 
assembled  and  factoiy  testing  is  now  in  progress..  Because  of  the 
urgent  need  for  the  NEW  system,  the  development  and  acquisition 
contract  was  structured  for  concurrency.  The  terms  and  conditions 
of  the  contract  stipulate  that  the  release  of  funds  for  production 
will  be  predicated  on  satisfactory  demonstration  of  system  performance 
capability.  The  procuring  agency  and  contractor  have  agreed  on  the 
portion  of  the  total  test  program  which  must  be  accomplished  prior  to 
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production  release.  It  was. decided  that  the  formal  reliability  test 
required  to  verify  achievement  of  the  contractual  quantitative  require¬ 
ment  could  be  deferred  until  after  production  commitment i  However,  an 
assessment  of  the  inherent  reliability  designed  into  the  NEW  system 
must  be  made  to  determine  if  a  minimal  acceptable  level  has  been 
achieved.  If  a  threshold  value  cannot  be  reached,  it  will  be  necessary 
to  redefine  the  system  requirements  ar.d  initiate  a  redevelopment  effort 
for  an  alternative  replacement  for  the  OLD.  system.  This  contingency 
system  will  be  designated  XYZ.  Nov  that  the  foundation  has  "been,  laid 
for  this  case  study,  the  remaining  presentation  will  adhere  to  the 
established  blueprint  for  solving  decision  problems. 

Definition  of  the  Problem 

The  primary  objective  of  the  designated  procuring  activity  is  the 
timely  introduction  of  a  cost-effective  means  of  satisfying  the 
operational  need  for  sufficient  warning  of  an  advancing  aggressor. 
Although  the  NEW  system  is  based  on  existing  technology  to  a  great 
extent,  it  does  contain  some  innovative  features  which  constitute 
technical  risks.  There  are  many  component  types  in  the  NEW  system 
which  are  state-of-the-ar.t  devices  and  thus  have  questionable  reli¬ 
ability  characteristics.  Although  there  are  other  uncertainties 
associated  with  the  NEW  development  effort,  it  will  be  assumed  that 
they  will  be  addressed  separately.  Thus  the  problem  will  be  defined: 
"From  a  reliability  viewpoint,  determine  the  advisability  of  committing 
funds  for  production  of  the  NEW  system."  Thus  A  is  the  set  of  actions 
that  a  decision  maker  might  take  regarding  the  release  of  production 
funds.  For  illustrative  purposes,  the  following  choices  are  assumed: 


35 


GSE/ittTH/66-il 

*•  -  *  %  * '  • 

=  Release  the  funds; -proceed  into  production. . 

&2  -  Wait  three  months  Cor  results  of  perf oraance  testing 
in:  progress. 

a*  =  Wait  six  months  for  results  of  formal/reliability  test. 

a^  =  Cancel  program;  initiate  development  of  the  contingency 
system. 

To  evaluate  these  options,  the  technical  risks  associated  with  NEW 
system  reliability  must  be  evaluated. 

Initial  Assessment  of  Uncertainties 

The  unknown  quantity  in  this  decision  situation  is  the  true  system 
reliability.  So  in  this  case  specific  values  for  the  system  failure 
rate,  X  ,  will  constitute  the  possible  states  of  nature.  The 
contractual  requirement  for  the  NSW  system  is  10,000  failures  per 
million  hours,  fpmh.  To  assess  the  probability  that  the  inherent 
reliability  is  less  than,  equal  to,  or  greater  than  this  specified 
value,  it  is  necessary  to  calculate  a  point  estimate  prediction  \and 
address  the  variability  associated  with  it.  At  the  critical  design 
review  for  the  NEW  system,  the  information  presented  in  Tables  IV 
and  V  was  presented  by  the  contractor. 
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.Table  JV 

REW  System  Reliability  Predictions 


1  * 


Table  V 

Parameters  for  a  System  Prior  Distribution 


Item 

Qty 

Per 

Sys 

ni  . 

ai 

K  -  M 
i  ~  ^ui 

a. 

Mi-— 

bi 

(X10“6) 

Vi 

(X10“$) 

•f-  Vi 
% 

■  (X10‘?) 

A 

1 

,40  ' 

*  50  . 

8000 

8000 

16000  / 

B 

2 

;i.20 

00 

VjJ 

1435 

2870 

344 

\ 

C 

5 

;i.3o 

i.240 

1050 

5240 

423 

D 

i  ; 

9.00 

29100 

310 

310 

1 

E 

2 

.35  - 

:  133 

2630 

5260 

3957’ 

> 

F; 

t  N 

4* 

# 

420 

950 

0 

O' 

227 

« 

22,630 

20,952 

,  g. 

\  Unit/Subsystem 

Failure  Rates 
(X10"6) 

Pr 

(\.  <  x-  \ 

XPi 

Xui 

\  -  nu) 

A 

Array  Antenna 

2500 

:  10000 

.75 

B 

Beam  Selector/Steerer 

750 

3000 

.90 

C 

Control/Display  Console 

700 

2100 

.90 

D 

Data  Processor 

300 

600 

.99 

E 

Emitter/Detector 

600 

3000 

.75 

F 

Frequency.  Randomizer 

■  0 

0 

u\ 

-  5000 

.90 
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These  tabulated  data  can  be  \ised  to  determine  estimates  for  the 
system  failure  rate  mean  value’,  and  any*  upper  bound,  ^  ,  from 


the  following  relationships, 

i  =  A 


(5-1) 


(5-2) 


(5-3) 


where  Z  ~  Normal  deviate  for  the  one  sided  confidence  level  of 


interest  (e.g.,  for  u  =  .90,  Z=  1.282) 

However,  in  decision  problems  there  are  usually  several  ranges 
of  interest  instead  of  just  two  (i.e.,  the. -intervals  of  o  <  A  <  4, 
andjQ,<  A  <oo).  In  this  example  there  will  be  six  P(  #j)  =  P(  A^) 
divisions, 

0<  X,£0.CC3 
0.003 <  X2  £  0.C05 
"0.005<  X,  £  0.015 
0.015<  X4  £  0.045 
0.045<  X5  £0.055 
0.055 <  X6  <  oo 

To  determine  the  P( X  j)  it  is  convenient  for  later  comparison  to 
assume  that  the  system  failure  rate  distribution  is  also  Gamma.  A 
Gamma  distribution  can  be  transformed  to  a  Chi-squared,  X^,  distribu¬ 
tion  with  d  degrees  of  freedom  by  lettir.g  d  =  2 (a  +  1)  and  X^(d)  =  2b  . 
(Ref  153:31).  In  this  case  the  values  ag  and  bs  for  the  system  failure 
rate  distribution  can  be  determined  from  ^  and  since 

S  3 ' 
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and  b*“  *%/  *t  •  "7-*n  X-  table-  can  bo  u«ed  to  rir.d 
P(x^,d)  =  F(  X  S  Xy  ) .  Tho  p(  X  j)  con  bo  determined  from  tho  relation¬ 
ship.  P(  Xj)  *  P(X^  pd)  ar.d  P(xJ,d)  -  0.  For  example,  using  equations 
(5-1)  «nd  (5-2)  and  the  data  listed  in  Table  V,  P(  X^)  can  be  computed 
as  follows, 


s  (.022630) 2 

.00020952 

=  2.46 


.00020952 
=  108 


*2  "  2bs  *u2 
=  2(108)  (.005) 

=  1.08 

P(x§,d)  =  P(1.08, 2(2.46)  +  2) 
-  Pd.08,6.9'2) 

=  .0070 

$  =  2(108)'(.015) 

=  3.24 

P(x^,d)  =  P(?.24,6.92) 

=  .1331 
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P(X3)  =  P(j^,d)— P(4fd)  ~- 
=  .1381  -  .0070 
=  .1311 

The  above  sequence  was  repeated  for  the  other  P( X  j)  and  the  results 
are  listed  in  Table  VI  along  with  assigned  point  values,  Xaj,  for  each, 
interval  which  will  be  used-  in  subsequent  calculations. 

Table  VI 


Values  for  P( X j)  Based  on  Predictions 


j 

LUa-Uffli 

?(x],2as  +  2) 

mom 

Xaj 

D 

M 

.0014 

.0014 

.002 

B 

1.08 

.0070 

.0056 

.004 

H 

3.24 

.1381 

.1311 

.010 

B 

9.72 

.7936 

.6555 

.030 

5 

11.88 

.8959 

.1023 

o 

O 

• 

oo 

1.0000  _ 

.1041 

.060 

Comparison  of  Consequences 

Now  that  all  prior  knowledge  concerning  KEaT  system  reliability 
has  been  quantified,  the  impact  of  the  various  X j  on  each  of  the  a^ 
can  be  determined.  In  this  <case,  the  primary  concern  is  to  minimize 
anticipated  life  cycle  costs  since  it  has  been  assumed  that  the 
performance  effectiveness  of  the  NEtf  system  is  acceptable.  There  are 
many  ways  to  estimate  life  cycle  costs,  most  of  which  are  rather 
elaborate  and  time  consuming  if  any  reasonable  degree  of  precision 
is  desired.  The  essential  elements  consist  of  development  and 
acquisition  costs  and  operation  and  maintenance,  G&M,  expenses.  The 


GRE/toATH/66-11 


factors  which  relate  to  thesfc  basic  costs  for  each  of  the  systems  in 
question  are  provided  in  Table  VII.  The  most  significant  portion  of 
total  life  cost  is  the  maintenance  expense.  Table  VIII  contains  the 
costs  of  monthly  maintenance  (assuming  $5CC0  per  repair)  for  each 
assigned  failure  rate,  Aai,  and  various  system  quantities. 
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Table  VII 

Life  Cycle  Cost  Factors 


"  - — System 

Factor'  7'--*^^ 

OLD 

NEW 

XYZ 

Prototype  Costs,  P 

- 

$  50M 

$  75M 

Price  per  System,  A 

- 

$  2M 

$2.5M 

Delay  Costs/Quarter,.  D 

- 

$  1M 

r 

Cancellation  Costs,  Cj 

- 

$0-75M  * 

O&M  (Cost  per  Repair),  R 

%  5K 

$  5K 

$  5K 

Observed/Specified  A  , 

.05 

.01 

Number  of  Systems,  N 

60 

60 

60 

‘Delivery  Rate  (Systems  Per  Month) 

- 

5 

5 

Development  Time  (Months) 

0 

24 

Production  Lead  Time  (Months) 

- 

15 

.12 

*  Cx  =  75M;  C2  =  50M;  C3  =  25M;  c4  =  =  C6  =  0 
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Table  VIII 

Monthly  Maintenance  Expense,  (in  the  millions  of  dollars) 


Num 

of 

System  Failure  Rate  X 

Sys 

N 

.004 

.01 

.03 

.05 

.06 

5 

.036 

.072 

:  .18 

.54 

.9 

1.08 

10 

.072 

.144  : 

.36 

1.08 

,  1.8 

2.16 

< 

15 

.10? 

.216 

.54 

1.62 

2. 7 

3.24 

20 

.144 

.288 

.72 

2.16 

3.6 

4.32 

25 

.180 

.360 

..90 

2.70 

4.5 

5.4 

30 

.216 

.432 

1.08 

3.24 

5.4 

6.48 

35 

.252 

.504 

1/26 

3.78 

6.3 

7.58 

40 

.288 

.576 

1.44 

4.32 

7.2 

8.64 

45 

.324 

.648 

1.62 

4.86 

8.1 

9.72 

50 

.360 

.720 

1.80 

5.40 

9.0 

10.80 

55 

.396 

.792 

1.98 

5.94 

9.9 

11.88 

60 

.432 

.864 

2.16 

6.48 

10.8 

12.96 
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Another  important  consideration  in •=  this  particular  case  is. the  impact 

*  r  _  „ 

that  additional  time  has  on  effecting  reliability  improvements. 
-Potential  enhancement  of  reliability  is  dependent  on  many  factors, 
including  current  technology,  available  resources,  physical,  space, 
etc.  There  is  no  universal  reliability  growth,  model  that  applies  to 
all  types  of  equipment  in  any  stage  of  development.  For  illustrative 
purposes,  a  simple  linear  relationship  of  estimated  reliability 
improvement  versus  time  is  considered  sufficient,  .To  account  for  the 
additional  time  advantage  in  subsequent  calculations,  an  estimated 
reliability  improvement  factor,  I  (M,  X)  will  be  used.  Values  for 
I  (M,  X)  for  M  =  0-30  months  and  X  =  .004,  .01,  .03,  .05  and  .06  ‘can 
be  obtained  from  Figure  1.  For  simplification,  I  (M,  .002)  =  1. 

Now  that  all  the  pertinent  information  arid  relationships 
regarding  'individual  cost  elements  for  both  existing  and  planned 
systems  have  been  discussed,  the  expressions  for  computing  the  Cij  can 
be  presented.  Specifically,  the  Cij  represents  total  life  cycle  costs 
for  a  ten-year  period  commencing  with  scheduled  production  release. 

The  delivery  rates  are  such  that  a  mix  of  existing  and  replacement 
occurs  only  during  one  year  starting  with  the  .first  increment.  During 
this  year,  there  is  an  average  of  30  each  present  and  replacement 
systems.  For  the  alternative  decisions,  the  corresponding  consequences 
are: 
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C  *  Pn  ♦  60 An  ♦  15fc(gfr,  .05)  +  12R(3Q,.05)  +  ' 

lj  ’  "  • 

12R(30,A4j)  +  105R(60,A4j) 

C2j  =  Pn  +  60An  +  D,+  18R(60,..05)  +  12R(30,.05)  + 
12R(30,  Aaj)  +  102R(60,,A4j) 

KJ.k.j) 

»  -« 

Cjj  -  Pn  +  -60 An  +  2D  *  21R(60,.05)  +  12R(30,.05)  + 

.  12R(30,  Aaj)  +  99R(60,  A^) 

1(6,  X,j)  • 

*  .«  ' 

=  Pn  +  Px  +  60Ax  +  36R(60,.05)  +  12R(30,.C5)  + 

12R(30,  Aaj)  +  84R{60,Xaj) 

1(24,  A-ftj) 

« 

Thus,  from  equation  (5-6),  G^-j  (in  ssilliona  of  dollars)  can  be 
determined  as  follovs, 

C-,-,  =  50  +  120  +  2  +  21(10.8)  +  12(5.4)  4 


p2(l.O?)  +  84  (2. 10  j 


(5-4) 


(5-5) 


(5-6) 


(5-7) 


=  670 
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The  remaining  C  jj  were  calculated,  in  a  similar  manner  .and  the 

*  *  •*--  -**  « 

results  are  listed  in  Table  IX.  Since  the  C^j  are  expressed  in 
quantitative  terms,  assignment  of  utility  measures  is  unnecessary. 


Table  IX 


Production  Release  Decision  Matrix 
(C^j  in  millions  of  dollars) 


System 

^VFailure 

D*cisio$NJlate 

x2 

*3 

*4 

*5 

*6 

a^  Proceed 

445  . 

493  - 

624 

1116 

1596 

1836 

a2  Wait  3  oos. 

477 

521 

-  652 

1039 

1363 

1498 

tij  Wait  6  mos. 

509 

551 

670 

987 

1220 

1315 

a^  Cancel 

843 

846 

.893 

994 

1053 

1072 

Qalculatlon  of  Expected  Value 

The  solution  to  this  production  release  decision  problem  is  to 
take  the  action,  a^,  which  is  most  likely  to  result  in  the  lowest 
life  cycle  cost.  This  selection  can  be  made  by  comparing  the  expected 
va1ue  of  each  alternative,  E(ai) .  The  E(a^)  can  be  computed  from  the. 
data  in  Table  IX  by  using  equation  (2-1) , 
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--  «  ■  «  *  * 

For  example,.  «' 

<=<«!>  =  £  “u  P<*j>  -  .  " 

J-A 

=  (445)(.00U)  +  (493)  (.0056)  + 

(624) (.1311)  +  (1116) (.6555)  + 

(1596) (.1023)  +  (1836) (.1041) 

=  $117114  (5-9) 

Similarly,  the  other  E(a^)  were  computed  and  the  following  values  were 
obtained  for  E(a2) ,  E(a^) ,  E(a^)  respectively:  $1069M,  $1CG0M,  $994M. 
Thus,  the  logical  decision  based  on  predicted  reliability  data  only 
is  a^  or  cancel  the  NEW  program  and  initiate  procurement  of  the  XYZ 
system. 

Second  Assessment  of  Uncertainties 

However,  in  addition  to  these  analytical  predictions,  logs  of 
individual  unit/subsystem  operating  times  and  failures  have  been 
maintained  as  portions  of  the  NEW  system  have  progressed  through 
various  phases  of  testing.  The  specific  information  reported  is  listed 
in  Table  X. 


Table  X 

•  Cumulative  Unit/Subsystem  Test  Data 


Unit/Subsystem 

A 

B 

c 

D 

E 

F 

Total  Failures  f^ 

5 

1 

4 

0 

1 

1 

Total  Time  t^ 

1500 

2000 

.  4750 

900 

2400' 

1500 
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.These  data  also  reflect  a -measure  of  system  reliability  and  may 
be  used  independent  of  the  prior  information  to  determine  estimates  for 
P(  X  j) .  Again  the  x2  distribution  may  be  used  by  letting  d  =  2(f  +  1) 
and  X^(d)  =  2tXuj.  If  it  is  assumed  that  the  data  in  Table  X  represent 
f  =  12  failures  in  approximately  t  =  1000  equivalent  hours  of  system 
operation,  then  the  ;p(  X j)  can  be  determined  using  the  same  procedure 
as  before.  These  P(  X  j)  -  based  solely  on  the  observed  test  data  - 
are  listed  in  Table  XI.  When  these  probabilities  are  used  in  the  de¬ 
cision  matrix  (Table  L%,  the  values  for  E(ai)  are:  $755M,  $755M, 

$755M,  and  $920M.  Therefore  the  test  results  are  inconclusive  as  far 
as  a  definite  decision  is  concerned. 


Table  XI 

Values  for  P(  X  Based  on  Test  Data 


The  tost  data  may  also  be  combined  with  the  prior  information 
recorded  in  Tables  IV  and  V  to  produce  a  more  accurate  revised  system 
failure  rate  distribution.  The  parameters  and  statistics  of  the 
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posterior  distributions  will  be  distinguished' from  prior  values  by 
using  script  letters  and  bold  type.  The  posterior  parameters,  for  the 
unit/subsystem  failure  rate  distributions  are  =  aj  +  f ^  and 
®i  =  ^i  +  *i»  The  posterior  system  failure  rate  distribution  mean 
and  variance  are  vs  and  s|,  respectively.  In  this  particular  case, 
when  the  individual  unit/subsystem  "pseudo”  and  actual  failures  and 
operating  times  are  combined,  the  resultant  values  for  Ai  and  are 
those  listed  in  Table  XII.  The  values  for  vs  anJ  are  determined 
by  following  the  same  procedure  used  to  obtain  n3  and  <r|  •,  Likewise, 
the  system  Gamma  parameters,  As  and  Bs,  can  be  computed  the  same  way 
as  before.  These  quantities  may  then  be  used  to  determine  the 
individual  P( A  j)  from  expressions  associated  with  Bayesian  one  sided 
upper  confidence  limits  given  by  Nagy  (Ref  153:29). 
x]  (d)  =  2BS  Auj 

d  =  2AS  (5-10) 

Again,  since  the  Xuj  are  stipulated,  the  values  for  x‘j  can  be 
calculated  and  the  x^  tables  can  ^be  used  to  find  ?(x| ,2Ag) .  The 
values  for  P(  A  j)  which  result  from  these  computations  are  listed 
in  Table  XIII.  When  these  revised  estimates  are  used  in  equation 
(5-8)  to  compute  the  E(a^)  the  results  (in  millions  of  dollars)  are: 
652,  674»  688,  and  898.  Thus,  the  preferred  course  of  action  is 
clearly  a,  and  production  release  may  be  granted  without  delay.  Note 
that  this  decision  is  the  reasonable  choice  although  i*s  is  greater 
than  .the  failure  rate  specified  in  the  contract.  In  this  case,  it  has 
been  illustrated  that  the  planned  deployment  of  the  NEW  system  with 
an  estimated  reliability  of  approximately  only  £1%  the  required  level  . 
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1,  tore  edit  effective  t hah  the'cmtipued.  use  of  the 
the  three  year  delay  required  for  the  TiZ  aySteB. 


OLD  system  or 


\ 

i 

i 


5.40  : 

1550 

3484 

3484 

2.20 

2835  ' 

.  776  , 

-  1552 

5.30 

5990 

885 

4425 

.  9.00 

30000 

300 

300 

1.35 

2533 

533 

;  1066 

1.40 

1920 

729 

729 

11,556 

Lues  for 

Tabl 

P(  X  j)  Ea 

e  XIII 

sed  on  Con 

ibined  Input 

15.95 

26.59 

79.77 

239.31 

292.49 

co 


r(x?,2jg 


.0000 

.0001 

.9425 

1.0000 

1.0000 

1.0000 


.0000 

.0001 

.9424 

.0575 

.0000 

.0000 


224; 

54. 

74.0 

l.p 

42.1 

38.0 
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VI,  Assessment  Technique  Analysis  and  Refinement 

The  sample  information  used  in  the  example  presented  was  obviously 
selected  to  illustrate  a  point.  There  is  absolutely  no  guarantee  that 
the  pooling  of  reliability  predictions  and  test  data  will  always  result 
in  such  a  drastic  change  in  the  preferred  course  of  action.  However, 
the  scheme  outlined  does  -produce  better  informed  decisions  which  are 
normally  less  costly  because  less  experimentation  is  usually  required. 

The  reason  for  this  is  better  appreciated  if  the  contributions  of  the  two 
inputs  are  analyzed., 

Examination-  of  Input  Contributions 

As  mentioned  in  Section  III,  the  quantification  of  prior  informa¬ 
tion  (in  this  case  reliability  predictions)  narrows  the  range  of 
exploration  for  the  true  state  of  nature  (system  failure  rate) ,  The 
outcomes  of  experimentation  (failure  and  time  data)  provide  a  further 
reduction  to  the  area  of  consideration  by  decreasing  the  variability 
of  the  initial  estimates.  These  contributions  are  apparent  when  the 
unit/subsystem  and  system  failure  rate  densities  are  examined.  In  the 
example,  the  greatest  amount  of  uncertainty  was  contributed  by  items 
A  and  E.  The  prior  and  posterior  failure  rate  density  functions  for  these 
two  items  are  plotted  in  Figures  3  and  4  respectively.  In  each  instance, 
the  posterior  variance  is  considerably  smaller  than  the  prior  variance. 

The  magnitudes  of  difference  can  be  determined  by  comparing  the  values 
for  «r ar5(j  4  2  (listed  in  Tables  V  and  XII)  for  these  two  and  the 
other  items.  '(Also  shown  in  Figures  3  and  4  are  g(X;  A,B)  for  other 
selected  test  data  for  comparative  purposes.) 
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The  values  of  fl’j^and  '  4  2  may  also  be  found  in  Tables  V  and  XII. 
■These  quantities  dictate  the  shapes  of  the  prior  and  posterior  system 
failure  rate  densities.  Again  the  posterior  variance  is  less  than  the 
prior,  and  therefore,  the  range  of  the  true  system  failure  rate  is 
smaller  which’ results  in  a  more  accurate  estimate  of  reliability. 
Another  factor  which  influences  the  accuracy  of  the  P(  X  j)  quantities 
is  the  .assumed  system  distribution. 

Gamma  Vs  Normal  System  Distributions 

As  previously  stated,  a  Gamma  system  failure  rate  distribution  was 
assumed  as  a  matter  of  convenience.'  Another  statistical  distribution 
which  is  widely  known  and  easily  applied  is  the  Normal  distribution. 
However,  in  order  to  employ  the  Normal  distribution  two  parameters 
must  be  specified  or  estimated.  To  analyze  the  sensitivity  of  the 
assessment  technique  used  in  the  example,  four  additional  sets  of 
sample  information  were  assumed  and  resultant  choices  compared  with 
original  decisions.  Rather  than  assuming  a  particular  value  for  the 
standard  deviation  of  the  hypothetical  test  data,  it  was  decided  to 
create  sufficient  information  from  which  a  sample  standard  deviation 
could  be  computed.  The  additional  sets  of  information  include  two 
unfavorable  sets  and  two  favorable  sets  as  compared  with  the  original 
example  set.  The  unfavorable  data  include  both  the  case  in  which  the 
same  number  of  failures  are  observed  in  half  the  time  and  the  case  in 
which  twice  the  failures  are  observed  in  the  same  amount  of  time.  The 
favorable  data  include  both  the  case  in  which  half  the  number  of 
failures  are  observed  in  the  same  amount  of  time  and  the  case  in  which 
the  same  number  of  failures  are  observed  in  twice  the  time.  The 
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sample  failure  information  for  these  four^nev  cases  plus  the  Original 
case  is  presented  in  Table  XIV. 

For  each  set  of  data,  a  set  of  P(  X/)  was  computed  twice  -  once 

assuming  a  Gamma ^system  distribution  and  again  assuming  a  Normal 

distribution.  Then  the  Normal  distribution  was  assumed,  the  sample 
2 

vs  and  *s  Were  calculated  from  the  following  relationships, 


=  i  t  a 

“T  4- 


<m 
m 

-2=E 


j=l 


s  ~pi  (1/tr  . 

m  -  1  (6-1) 

For  example,  using  the  data  in  Table  XIV  in  the  first  and  third- 
columns. 


V  (f ,  .  5t)  =_1  F_2  +  JL  +  J.  +  _1* 
S  12  [35  40  45  50^ 


=  .024253 


~|^2'(. 028571  -  .024253)2  +  5(. 025000  -  .024253)2 

-  .024253) 2] 


+  4(. 022222  -  .024253) 2  +  (.020000 
i:  .002605 


To  obtain  the  P(  X  j),  it  is  necessary  to  compute  Zj  -  (  Xuj  -  t«„)/  *s 
and  then  use  standard  statistical  tables  to  obtain  P(Zj)  -  P(X  ^  Xuj) . 

The  P(  X j)  can  than  be  determined  from  the  relationships  P( X j)  -  P(Zj)  - 
P(Zj_1)  and  P(ZQ)  =  0. 

The  results  of  recomputing  the  F.(a^)  for  each  set  of.  data  are 
listed  in  Tables  XV  and  XVI  when  Gamma  and. .Normal  distributions, 
respectively,  are  assumed  to  apply.  Analysis  of  these  -results 
indicates  that  the  assessment  technique  yields  the  same  decision  for 


57 


GRE/MATH/ 66- 11 


Table  XV 

E(a^)  Values  Assuming  Gamma  System  Distribution 


P(  Xj)  Data  Source 

E(af) 

E(a2) 

E(a3) 

Prior  Information  (Predictions) 

1171 

1069 

1000 

Unfavorable  Test  Data  (12/500) 

1101 

1027 

976* 

Unfavorable  Test  Data  (24/1000) 

_  11.11 

1039 

,  934* 

Example  Case  Test  Data  (12/1000)' 

755* 

755* 

755* 

Favorable  Test  Data  (6/1000) 

595* 

622 

.643 

Favorable  Test  Data  (12/2000) 

632* 

662 

'  685 

Unfavorable  Posterior  (12/50Q) 

1081 

1011 

964* 

Unfavorable  Posterior  (24/1000) 

1116 

1039  ! 

937* 

Example  Case  Posterior  (12/1000) 

652* 

.  674 

638 

Favorable  Posterior  (6/1000) 

617* 

645 

664 

Favorable  Posterior  (12/2000) 

■ _ 

650*  : 

680 

702 

^Minimum-.  E(a^) 
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Table  m 

E(a^)  Valuog  Assuming  Nonmtl  Systeu  Distribution 


P(  Aj)  ‘Data  Source 

Hsl 

Bl 

“(a->) 

?{*,} 

tv 

Prior  Information  (Predictions) 

963 

926 

891* 

962 

Unfavorable  Test  Data  (12/500) 

1116 

1039 

93'"*  ‘ 

G  V 

Unfavorable  test  Data  (24/1000: 

1098 

1024 

9~5* 

*  S'*  ; 

Example  Case  Test  Data  (12/1000) 

704* 

731  ! 

716 

Favorable  Case  Test  Data  (yiCOO) 

610* 

6ii8 

658 

'  *  99 9  < 

V  mS0 

Favorable  Case  Test  Data  (12/2000) 

661* 

680 

692 

^0 

Unfavorable*  Posterior  (12/500) 

107.9 

1010 

963* 

936 

Unfavorable  Posterior  (24/1000) 

1092 

;  ,1020 

972* 

090 

z1-  / 

Example  Case  Posterior  (12/1000) 

648* 

671 

685 

893  ' 

Favorable  Posterior  (6/1000) 

615* 

643 

662 

893  , 

Favorable  Posterior  (12/2000) 

619* 

.  647 

665 

or ~ 

✓ 

^Minimum  E(a^) 
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either  distribution  axcspt-vhen-only  the 'prior  information  if  Hied  to 

eoapute  the  P(  A  ) .  Alio  the  extent  of  discrimination  between  the 
•# 

K(a^)  computed  from  the  same  data  base  appears  to  be  remarkably  similar 
for  the  two  assumed  distributions.  Comparison  of  the  E(a^)  based  on 
unfavorable  vs  favorable  test  results  reveals  that  the  intuitive  de¬ 
crease  in  calculated  value  occurs  as  fewer  failures  are.  observed  ir. 
greater  time.  Overall,  the  assessment  technique  seems  realistic  and 
practical. 

Other  Practical  Considerations 

In  addition  to  the  features  of  the  technique  presented  with  the 
example,  and  discussed  above,  there  are  two  other  refinements  which  are 
considered  worthy  of  discussion.  One  pertains  to  the  consideration  of 
different  opinions  when  assigning  parameters  of  the  unit/subsystea 
prior  failure  rate  distributions.  The  other  relates  to  the  treatment 
of  different  types  of  test  data  when  determining  operating  times. 

Weighting  Prior  Assignments.  There  are  many  instances  in  which 
different  individuals  provide  inputs  for  the  various  pieces  of  equip¬ 
ment  that  make  up  a  system,  The  analyst  might  also  wish  to  combine 
thf  inputs  from  several  sources  in  ordor  to  formulate  &  single  overall 
estimate  for  ari  item  failure  rate.  Fox  advances  the  idea  of  assigning 
weighting  factors  to  each  contributor  (Ref  84:3).  These  weights  car. 
be  based  on  either  the  contributors  accuracy  and  consistency  of  pre¬ 
vious  predictions  or  his  extent  of  participation  if  a  joint  assessment 
is  required  (i.e.,  a  prior  based  on  combined  government,  consultant, 
contractor  inputs) .  In  either  case,  if  a  weighting  factor  $  is 
assigned  to  the  k*1*1  of  r  individuals ,  then  the  parameters  for  a 
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Ganna  prior  can  be  computed. ^rpm ..the  following  expressions, 

•  - 

k  -  1 

*>=  I-  *k\ 

k  =  1  “ 

Weighting  Heterogeneous  Test  Data.  The  other  weighting  factor 
considered  important  is  a  multiplier  which  accounts  for  the  difference 
in  severity  of  the  various  types  of  tests  to  which  an  item  is  usually 
subjected.  The  concept  of  attaching  more  significance  to  data  ob¬ 
tained  under  more  difficult  environments  has  been  proposed  by  .Pczr.er. 
He  states  this  may  be  done  "by  weighting  the  time  .experience  in  par¬ 
ticular  environments  by  the  k  factor  corresponding  to  these  environ¬ 
ments.  .  These  k  factors  are-  environmental  failure  rate  acceleration 
factors  such  as  those  in  MIL-STD-756."  (Ref  162:139) .  Test  severity 
weights  need  not  be  of  the  magnitude  generally  associated  with 
acceleration  factors.  The  important  consideration  is  that  the  value 
computed  or  assigned  reflect  the  extent  of  additional  exposure  ex¬ 
perienced  by  the  equipment  beyond  that  normally  expected.  As  with 
the  reliability  predictions,  test  severity  weights  based  on  considered 
.-opinion  or  technical  Judgment  of  competent  engineers  may  be  used. 
Again,  if  several  subjective  inputs  are  to< be  combined,  the  con¬ 
tributors’  weights  can  be  included  in  deriving  an  overall  set  of  test 
severity  weights  for  the  various  types  of  tests  to  be  performed.  In 
general,  a  value  k  for  a  particular  test  can  be  determined  from  the 
following  expression, 
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where  Y.  =  contributors  weight 

kj  =  individual  test  severity,  weight 
r  =  number  of  contributors 

Note  that  the  term  used  for  the  contributor's  weight  is  different 
than  the  one  used  in  assigning  priors.  This  was  done  purposefully 
to  distinguish  between  judgements^  pertaining  to  reliability  predictions 
and  those' concerning  difficulty  of  test  environments. 

With  the  addition  of  these  two  weighting  factors  a  recap  of  the: 
revised  assessment  technique  for  the  general  case  is  in  order. 

Revised  General  Approach 

Tno  Bayesian  reliability  assessment  technique  may  be  implemented 
in  a  variety  of  situations',  It  is  not  necessary  for'  the  situations  to 
be  based  on  the  need  to  make  a  particular  decision  as  was  illustrated  < 
The  procedures  outlined  and  methods  presented  may  also  be  used  for 
periodic  determination  of  the  status  of  reliability  achievement.  In 
summary,  the  following  step-by-step  activities  are  required  to  obtain 
and  update  reliability  estimates  using  subjective  evaluation  and  Bayes 
techniques : 

(1)  Quantification  of  all  prior  knowledge  concerning  failure 
rate  predictions.  This  effort  may  be  a  singular  or 
combined  input.  If  combined,  contributor  weights  may 
be  used  to  obtain  single  values  for  prior  parameters 
for  calculation  purposes.  The  assertion  of  uncertainty 
must  be  carefully  considered  in  light  of  potential 
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Impact  when  combihad~vith  meaaured  data.  Parameters 
for  prior  distributions  may  be  determined  by  a  variety 
of  methods  and  Tables  I  and  II  should  be  beneficial. 

(2)  Determination  of  a  system  prior  distribution.  This  may 
be  accomplished  by  summing  the  means  and-  variances  of 
constituent  element  failure  rate  distributions.  If  only 
a  one-sided  upper  confidence  limit  for  the  system  failure 
rate  is  of  interest,  equation  (5-3)  may  be  used.  If 
discrete  intervals  are  desired,  then  either  a  Gamma  or 
Normal  system  distribution  may  be  assumed  and  particular 
probabilities  may  be  computed  as  illustrated. 

(3)  Collection  of  time  and  failure  data.  This  is  the  most 
critical  activity  associated  with  any  estimation  task. 
Posterior' estimates  are  only  meaningful  if  they  are 
based  on  accurate  and  complete  information.  In  order 
to  obtain  high  quality  data,  special  emphasis  must  be 
placed  on  the  proper  training  and  motivation  of  personnel 
responsible  for  maintaining  records.  There  is  also  the 
question  of  which  types  of  anomalies  to  consider  and  which 
ones  to  censor.  The  definition  of  relevance  is  many  times 
a  subject  for  negotiation  if  the  products  of  the  assess¬ 
ment  effort  are  used  for  acceptance  purposes.  Also,  test 
times  may  be  adjusted  to  reflect  severity  of  equipment 
exposure.  This  should  be  done  before  tests  are  started 

to  avoid  undue  bias  based  on  outcomes. 
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‘  (4)  '  Determination  ofa.  system  posterior  distribution.  This 

involves  adding  the  constituent  element  ’'pseudo"  failures 
and  times  with  corresponding  observed  values  to  modify 
the  prior  parameters.  System  reliability  indices  can 
be  computed  from  the  posterior  distribution  in  the  same 
manner  as  fran  the  prior  distribution. 

(5)  Analysis  of  sensitivity.  This  optional  task  may  be 
performed  if  there  is  doubt  concerning  the  impact  that 
a  particular  quantity  has  on  the  overall  results.  If 
more  precision  is  desired  than  that  achievable  from 
assuming  either  a  Gamma  or  Normal  distribution,  then 
Monte  Carlo  techniques  can  be  employed  to  determine 
exact  confidence  bounds. 

The  tasks  outlined  above  are  quite  general  and  are  considered 
to  be  useful  for  widespread  applications.  To  facilitate  the  recording 
of  the  inputs  necessary  for  a  Bayesian  assessment  effort,  sample 
worksheets  are  included  as  Appendix  B. 
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VII.  Conclusions. .and  Recommendations 


Conclusions 

It  has  been  shown  that  Bayesian  statistics  used  in  conjunction 
with  decision  theory  offer  a  suitable  framework  for  solving  cost 
effectiveness  type  problems  involving-,  uncertainty.  When  the  uncer¬ 
tainty  is  reliability,  there,  is  considerable  advantage  to  be  realized 
in  combining,  predictions  with  test  data  to  obtain  greater  precision  in 
the  reliability  estimate.  Therefore*  it  is  concluded  that,  the 
implementation  of  Bayesian  techniques  in  the,  solution  of  reliability 
decision  problems  can  produce  more  conclusive  results  with  the  rdded 
advantages  of  being  economically  practical  and  intuitively  appealing. 
Recommendations 

Based  on  the  findings  of  this  study,  it  is  recommended  that  the 
reader  desiring  to  further  pursue  the  field  of  Bayesian  statistics 
consider  the  following  topics: 

(1)  Development  of  simple^  algorithms  and  other  aids  to 
establish  prior  distributions  based  on  predictions. 

(2)  Investigation  of  other  closed  distribution  forms  to 
fit  the  -system  failure  rate. 

(3)  Development  of  a  simple  generalized  Monte  Carlo  model 
which  can  be  used  to  determine  exact  system  failure 
rate  bounds. 

Also,  it  is  suggested  that  this  study  be  used  as: 

(1)  an  aid  in  the  study  of  Bayesian  statistics  and  decision 
theory; 


(2)  as  a  source  of  references' and  possible  thesis  topics 
for  future  Air  Force  institute  of  Technology  students. 
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